1. Field
The present disclosure relates to communications in computer networks. More particularly, this invention is directed toward network switching with layer 2 switch coupled co-resident data-plane and network interface controllers.
2. Description of Related Technology
In modern communications networks, information (voice, video, data) is transferred among the nodes of the communications network in a form of packets. A packet is a formatted unit of data carried by a packet-switched network. A packet consists of two kinds of data: control information and user data (also known as payload). The control information provides data the network needs to deliver the user data, for example: source and destination network addresses, error detection codes, sequencing information and other control information known to a person of ordinary skill in the art. Typically, control information is found in packet headers and trailers, with payload data in between.
In traditional computer systems, a computer is connected to a communication network via a network interface controller (NIC). The NIC comprises a network facing physical interface to connect to the communication network, an electronic circuitry required to communicate using a specific physical layer and data link layer standards, and an internal facing physical interface allowing a driver to interface the NIC with the computer. The electronic circuitry may comprise an application-specific integrated circuit (ASIC). An ASIC comprises an integrated circuit (IC) customized for a particular use, rather than an IC intended for general-purpose use.
The packet processing at the NIC is interrupt driven. Interrupt is a signal to the computer's kernel emitted by hardware or software, indicating an event that needs immediate attention. Considering, by means of an example, an incoming packet arriving at the inbound network facing physical interface of the NIC. The NIC writes the packet into the NIC's memory; ascertains that the packet is valid by performing verification, e.g., a Cyclic Redundancy Check (CRC), and then sends the packet to the computer's memory buffer requested by a driver from and allocated by a kernel. A kernel is a computer program that manages input/output requests from software entities, and translates the requests into data processing instructions for the central processing unit and other electronic components of a computer. The kernel is a fundamental part of a modern computer's operating system. After the computer's memory buffer has been allocated, the driver provides the address of the computer's memory buffer to the NIC. After writing the packet to the computer's memory buffer, the NIC sends an interrupt to the kernel. The kernel responds by servicing the interrupt, with all the overhead associated with the interrupt, i.e., suspending current activities, saving the current activities state, and executing an interrupt handler routine. This interruption is temporary, and, after the interrupt handler routine finishes, the processor resumes normal activities.
The interrupt handler routine carries out packet processing related to packet management, e.g., switching/routing, packet/frame discrimination, Quality of service (QoS) enforcement, access control, encryption, Transmission Control Protocol (TCP) offload processing, and other packet management known to a person skilled in the art, and delivers the packet to the destination entity.
The packet management is implemented in the kernel's software, and since the kernel software runs on a general purpose central processing unit, the software implementation has low performance. Coupled with the interrupt overhead, the packet processing using the NIC negatively affects network performance, e.g., throughput, latency, and other network performance criteria known to a person of ordinary skill in the art.
Based on the foregoing, usage of a traditional computer systems for general packet management, is inefficient. To solve at least some of the problems, network processors, which comprise software programmable hardware devices, often implemented as IC, have been developed. Network processors have specific features or architectures optimizing packet processing within the communication networks. Such packet processing may comprise e.g., pattern matching, key lookup, data bit-field manipulation, queue management, control processing, and other processing known to a person of ordinary skill in the art.
Using the specific function of the network processor, the software program implements an application that the network processor executes, resulting in the piece of physical equipment performing a task or providing a service. Such applications may comprise, e.g., switching/routing, packet/frame discrimination, Quality of service (QoS) enforcement, access control, encryption, Transmission Control Protocol (TCP) offload processing, and other applications known to a person skilled in the art.
In addition, network processors use data-plane architecture that defines the processing of the packets arriving on an inbound interface to determine the path from the internal data-plane fabric to the proper outgoing interface(s). To support such architecture, the network processors employ event driven processing. An event is an action or occurrence detected by a system entity that then transfers the event to another entity that is a consumer of the event. The consumer entity eventually applies a reaction to the event. Unlike an interrupt that may demand immediate service at any time, events are normally handled synchronously, i.e., the system explicitly waits for an event to be serviced (typically by calling an instruction that dispatches the next event). Since the event is serviced after a previous event has finished, the overhead associated with interrupt processing is avoided.
Considering, by means of an example, an incoming packet arriving at the network processor network facing inbound interface. The inbound interface writes the packet into a memory; ascertains that the packet is valid by performing verification, e.g., a Cyclic Redundancy Check (CRC) and then sends the packet to the computer's memory buffer requested by the interface and allocated by a memory manager. After the computer's memory buffer has been allocated, the memory manager provides the address of the computer's memory buffer to the processor network inbound interface, which writes the packet to the computer's memory buffer. After writing the packet to the computer's memory buffer, the inbound interface generates an event. The event is provided to an event handling entity, e.g., via a scheduler, that schedules the event processing by the packet destination entity.
When the destination entity is ready to carry out the processing of the packet, the entity requests an event from the event handling entity. The event handling entity then provides the event caused by the packet, and the destination entity carries out the packet processing.
Optionally, the event handling entity may carry out processing related to packet management, e.g., packet/frame discrimination, Quality of service (QoS) enforcement, access control, encryption Transmission Control Protocol (TCP) offload processing, and other packet management known to a person skilled in the art, and delivers the packet to the destination entity. Since the packet management is implemented using the specific function(s) of the network processor, the implementation has high performance.
Current computer systems increasingly employ virtualization, i.e., a process by which a virtual version of computing resources, such as hardware and software resources, i.e., a central processor unit, a storage system, an input/output resources, a network resource, an operating system, and other resources known in the art, are simulated by a computer system, referred to as a host machine. A typical host machine may comprise a hardware platform that optionally together with a software entity, i.e., an operating system, operates a hypervisor, which is software or firmware that creates and operates virtual machines, also referred to as guest machines. Through hardware virtualization, the hypervisor provides each virtual machine with a virtual hardware operating platform. By interfacing with the virtual hardware operating platform, the virtual machines access the computing resources of the host machine to execute virtual machines' respective operations. As a result, a single host machine can support multiple virtual machines, each operating an operating system and/or other software entity, i.e., an application, simultaneously through virtualization.
FIG. 1 depicts a conceptual structure of a virtualization system 100. A hardware platform 102, comprises all physical entities embodying computing resources required by a specific host machine, i.e., a central processor unit, an input/output resources, a storage system, a network resource, and other resources known to a person having ordinary skill in the art. To avoid undue complexity, only a storage system 104, a network resource 106, are shown. The storage system 104, may comprise a hard drive, a semiconductor based memory, and other types of memory known to a person of ordinary skill in the art. The terms storage system and memory are used interchangeably. The network resource 106 may comprise at least one NIC.
The hardware platform 102, together with an optional software entity 108, i.e., operating system, comprises a host machine operating a Type 2 hypervisor, also known as hosted hypervisor 110. As well known to a person of ordinary skill in the art, the optional software entity 108 is not necessary for Type 1 hypervisor, also known as native hypervisor. A hypervisor is software or firmware entity that creates and operates at least one virtual machine, also referred to as a guest and/or a guest machine. As depicted in FIG. 1, the hosted hypervisor 110 created and operates three virtual machines 112; however different number of virtual machines, including a single virtual machine, is contemplated. Through hardware virtualization, the hosted hypervisor 110 provides each virtual machine 112 with a virtual hardware operating platform. By interfacing with the virtual hardware operating platform, the virtual machines 112 access the computing resources of the host machine to execute the virtual machines' respective operations. As a result, a single host machine can support multiple virtual machines 112, each operating an operating system and/or other software entity, i.e., an application, collectively shown as 116. simultaneously through virtualization. Parameters configuring operation of the virtual machine 112 are defined via structure 114. In an aspect, the structure 114 may comprise at least one register.
To enable transfer of packets into and from the virtualization system 100, via network resource 106, as well as routing the packets among different entities of the virtualization system 100 via internal network, using interrupt driven kernel packet management may degrade network performance. Additionally, divorcing the packet routing and management functionality from the kernel's functionality enables the routing and management functionality to be developed and provided without dependence of a specific kernel's implementation.
One possible approach to the above-identified problems is disclosed in an application Ser. No. 14/542,485 entitled “NETWORK SWITCHING WITH CO-RESIDENT DATA-PLANE AND NETWORK INTERFACE CONTROLLERS,” filed on Nov. 14, 2014. The application discloses a network interface apparatus, which implements packet management, including layer 2 switching, layer 3 switching, as a software executing at a co-processor of the network interface apparatus. Consequently, although such arrangement has an advantage in incurring no hardware cost in term of, e.g., additional chip area, power consumption, the packet management may not be as fast as a full hardware implementation of some of the packet management functionality, e.g., the layer 2 switch. The hardware implementation may comprise, e.g., an Application Specific Integrated Circuit (ASIC).
Furthermore, entities of the network interface apparatus, i.e., the NIC, the packet input processor, a packet output processor, and a medium access controller, are communicatively coupled via hardware loopback entities, which limit flexibility of incoming and outgoing packet switching/routing.
Accordingly, there is a need in the art for a network switching, providing a solution to the above identified problems, as well as additional advantages.